With 3D televisions where a user can perceive stereoscopic images, naked-eye 3D television where the user can view stereoscopic images without wearing glasses has started to come into practical use. With naked-eye 3D television, the display face has, for example, lenticular sheets or parallax barriers (disparity barriers), so as to control images entering the left eye and the right eye by the viewing position. That is to say, the configuration is such that left eye images and right eye images are generated, with the left eye images being observed with the left eye alone, and right eye images being observed with the right eye alone.
By using such technology, crosstalk where images entering the left eye and right eye become mixed is suppressed, and stereoscopy is enabled even without wearing glasses.
However, with such methods, correct stereoscopy can only be obtained at a restricted viewing position as to the display. Accordingly, in the event that the observation position of the user is a different position from the stipulated position, pseudoscopy, where the right eye image enters the left eye and the left eye image enters the right eye, or crosstalk, where the right eye image and left eye image become mixed, occur.
To solve this problem, there has been proposed a configuration in which, in addition to the left eye image and right eye image corresponding to the one normal observation point, an image is generated from a new viewpoint of a setting where crosstalk does not occur when viewed from another observation position. That is to say, image display with suppressed pseudoscopy and crosstalk is performed with the left eye image and right eye image corresponding to various observation positions of the user being selectable in accordance with the observation position of the user.
Specifically, based on images of two viewpoints that are originally input to the display device, i.e., on the two viewpoint images of the left eye image and the right eye image, viewpoint images other than these two viewpoint images are further generated. Two optimal images corresponding to the user observation position can be matched from the original left eye image and right eye image, and further the generated artificial viewpoint images, in accordance with the observation position of the user as to the display, and displayed, whereby display and observation with suppressed crosstalk where the right eye image and left eye image become mixed is enabled.
A specific example of processing will be described with reference to the drawings.
FIG. 1 represents an image obtained in a case of shooting subjects A and B from two different viewpoints. As illustrated in the drawing, a left image 21 shot with a left camera 11 and a right image 22 shot with a right camera 12 differ with regard to the position of the subjects, in accordance with the distance from the cameras, with the subject B being hidden behind the image of subject A more in the right image 22 than in the left image 21.
FIG. 2 represent an image obtained in a case of shooting subjects from four different viewpoints (viewpoint 1 through 4). The subjects A and B are shot from the four cameras of viewpoint 1 camera 31, viewpoint 2 camera 32, viewpoint 3 camera 33, and viewpoint 4 camera 34.
The image shot by viewpoint 1 camera 31 is viewpoint 1 image 41,
the image shot by viewpoint 2 camera 32 is viewpoint 2 image 42,
the image shot by viewpoint 3 camera 33 is viewpoint 3 image 43, and
the image shot by viewpoint 4 camera 34 is viewpoint 4 image 44.
As illustrated in the drawing, change in the positions of the subjects increases among the images of viewpoints 1 through 4 as the number of viewpoints increases and the distance between cameras increases.
In the event that the inputs image to the image processing device performing 3D image display are only the two images corresponding to the viewpoint 2 and viewpoint 3, a multi-viewpoint image generating unit of the image processing device generates the viewpoint 1 image 41 based on the viewpoint 2 image 42 for example and artificially generates the viewpoint 4 image 44 using the viewpoint 3 image 43.
However, there exists in the viewpoint 1 image 41 a region not included in the viewpoint 2 image 42 (the region of the subject B that was hidden behind the subject A), and the multi-viewpoint image generating unit needs to interpolate this region by image processing. In the same way, there exists in the viewpoint 4 image 44 a region not included in the viewpoint 3 image 43 (the background region that was hidden behind the subject A), and this region needs to be interpolated by image processing. Normally, such a region is called an occlusion region.
Conventionally, with multi-viewpoint generating technology where two or more viewpoint images as used to generate an even greater number of viewpoint images, there has been proposed a technology in which image depth information having subject distance information, for each pixel or in increments of blocks, is generated from the input images, and different viewpoint images are generated based on depth information obtained from the image depth information.
With conventional techniques, the position of a subject obtained from the depth information is converted into a position on an image obtained from a different viewpoint position to be newly generated, thereby projecting the subject, and generating a new viewpoint image.
With such a technique, images can be generated from any viewpoint, but the image quality of new viewpoint images affects precision of depth information. Accordingly, depth detection processing is necessary to generate high-precision image depth information, and accordingly there has been a problem that the circuit scale becomes great.
Also, as described above, in the event that there is an occlusion region equivalent to a region that was hidden in the actually shot image, there are cases where image information, necessary for the viewpoint image to be newly generated, cannot be obtained from the pixel images. As a result, there is a problem that setting of the pixel values of the viewpoint image to be newly generated cannot be made, resulting in a hole in the image. Image interpolation technology is necessary to solve this problem, and there has been the problem that the necessity for a circuit for this interpolation processing also leads to increased circuit scale.